1. Field of the Invention
The present invention relates generally to the fields of image processing for correction of image distortion produced by a wide-angle lens or simulation of a wide-angle lens and the creation of navigable virtual tours comprising primarily of panoramic images or video. In one embodiment, the invention relates to a system permitting the creation of immersive spherical environment maps from the class of panoramic imaging systems characterized by a 360-degree lateral field of view and a vertical field of view that is usually less than 180 degrees and the creation of navigable virtual tours comprising primarily of such spherical environment maps. The invention permits the use of an arbitrary lens in the aforementioned class of lenses for capturing the source panoramic scenes. In this context, a virtual tour refers to an interactive environment consisting of one or several (possibly interconnected) panoramic scenes (video, still or rendered) usually represented as cylindrical or spherical environment maps.
2. Description of the Prior Art
When creating navigable virtual tours, it is desirable to use a panoramic image acquisition system that encodes as much visual information as possible in a single image frame and that does so without any moving parts. Several methods of acquiring panoramic images exist. One solution involves using an orientation-sensing device to sense the orientation of the user and then using the input to rotate a conventional camera to cover the field of view indicated by the user's orientation. Such a system is described in U.S. Pat. No. 4,728,839. Apart from the obvious handicap of not being able to permit more than one user to comfortably navigate the remote scene, the inherent delay between the transmission of the orientation signal and the positioning of the camera to coincide with the desired orientation severely limits the practical value of this system. A more elegant solution to the problem is to use a wide-angle lens system. The fish-eye lens system described in U.S. Pat. No. 5,434,713 remedies some of the limitations of the conventional lens and camera system.
One class of wide-angle lens system permits the entire 360-degree lateral visual space around the user to be imaged in a single image frame. Although the vertical field of view of such systems can in principle be up to 180 degrees (providing a completely immersive panoramic scene), most practical systems of this type usually have a vertical view field that is less than 180 degrees. In this disclosure, this class of lens systems will be referred to as panoramic annular lens systems since they produce a donut-shaped, annular panoramic image. Two examples of panoramic annular lens systems are disclosed in U.S. Pat. Nos. 4,566,763 and 6,459,451. Wide-angle imaging systems generally suffer from inherent (predictable) perspective distortion and so for the images they produce to be viewed comfortably by a human being, they need to be de-warped. Several solutions to the perspective distortion problem exist. One such solution is disclosed in U.S. Pat. No. 5,185,667. The '667 patent describes a system utilizing complicated trigonometric transformations and assuming a mathematically ideal fish-eye lens. The solution described in U.S. Pat. No. 6,005,611 remedies some of the limitations of the system presented in the '667 patent. The 611 patent uses control points obtained via a set of projection equations and a global bivariate polynomial to correct the perspective distortion of the fish-eye lens.
Several products (including Apple Computer's QTVR Authoring System) utilize image-stitching techniques to stitch several (possibly overlapping) conventional images in a tedious and error-prone (due to disparities in the characteristics of the individual image segments used) procedure to create cylindrical or spherical environment maps for the creation of virtual tours. U.S. Pat. No. 5,396,583 discloses a perspective-correction algorithm for cylindrical environment maps.
The system disclosed in U.S. Pat. No. 5,990,941 remedies some of the limitations of the image-stitching methods by using one or two fish-eye images to create spherical environment maps. The system described in the '941 patent is also limited by the tediousness and inaccuracies of the image-stitching process. Furthermore, attempting to use the system described in the '941 patent to create spherical environment maps containing video would require the use of at least two video cameras and additional equipment to control and combine the output from the two video cameras.